A grating optical sensor is disclosed in WO 97/22 849. It is provided for accurately determining spatial and/or temporal spacings in focused image sequences of a lens/pupil system and/or determining spatial and/or temporal object parameters in real time such as, for example, speed or depth. A 3D grating has also already been used to carry out model calculations relating to the inverted retina of the human eye and to relate them to subjective phenomena known from human vision. In the preferred form, the 3D grating has a hexagonal structure. Other structures with centrosymmetrical diffraction patterns are, however, likewise possible.
Since the investigations of O. Lummer and the industrial development of daylight-like luminaires, it has been realized that there is an as yet unexplained resonance between sunlight and human vision. This has resulted in all the previous recommendations for approximating the spectrum of artificial light sources to the sunlight spectrum. In particular, in the case of color perception in phototopic day vision, there occur in the event of a change of illuminations having a different spectral composition of the radiation displacements of the color values which are compensated adaptively in human vision after a relatively short or, in part, relatively long time by means of approximate color constancy performances of the eye. The v. Kreis model, which attributes the adaptivity to the visual pigments of the retina, presently serves as an incomplete explanatory model for this. In addition, there are even more incomplete cortical explanatory models from other authors.
On the other hand, it has been documented many times that the phototopic seeing process cannot be characterized solely by the spectral light sensitivity of the individual cones. The very much more complex mode of operation of the visual sense requires knowledge of the luminance distribution in the entire visual field for the purpose of judging many visual tasks. Human vision is not based on the stimulus/reaction response of individual pixels. It takes account of the relative values over the entire field of view. In addition to chromatic adaptation effects, scattering of light at ocular media influences the extent of the achromatic axis (black-gray-white axis) centering the color space. It is therefore an illusion to believe that spectral photometers will be the ideal instruments of future chromatometry and color determinations, even if they are designed on the detection of overlapping RGB values. Likewise incomplete is a chromatometric technique which respectively dispenses with determining the triad of brightness/hue/saturation simultaneously and with reference to a entire field of view.
There is thus a growing need to have available in the future color sensors which can measure color values with reference to the spectral sensitivity curves of human vision, and ensure, given adaptation to artificial illuminations, an approximate color constancy corresponding to human vision. It is the object of the invention to create such a sensor.
The invention proceeds from the finding that it is possible, by inserting a diffractive multilayer (3D) grating into the image plane of an imaging lens/pupil system in the near field downstream of the grating (Fresnel/Talbot space; Fourier space or reciprocal grating), to make available three chromatic diffraction orders (RGB triple) with in each case six discrete interference maxima on mutually concentric circles, such as are described in the case of a hexagonal grating structure by means of the v. Laue equation known from crystal optics.
The v. Laue equation for diffractive space lattices requires for the production of constructive interference maxima the simultaneous satisfaction of the three phase conditions in the equation |1|
(cosxcex1-cosxcex1xc2x0)=h1xcex/gx
(cosxcex2-cosxcex2xc2x0)=h2xcex/gy
(cosxcex3-cosxcex3xc2x0)=h3xcex/gzxe2x80x83xe2x80x83|1|
(h1 h2 h3=triple of integral diffraction orders n; xcex1xc2x0, xcex2xc2x0, xcex3xc2x0=aperture angle of the light cone incident in the 3D grating, relative to x, y, z; xcex1, xcex2, xcex3=angle of the diffraction orders relative to x, y, z; xcex=wavelength; and gx, gy, gz=grating constant in the x-, y-, z-axial direction). Assuming a hexagonal packing of the optically diffracting elements and grating constant dimensions in xcexcm of gx=2xcex111, gy=4xcex111/3, gz=4xcex111, in equation |2|, xcex111 constitutes the wavelength diffracted with maximum transmission into the 111 diffraction order.                               λ          ⁢                      xe2x80x83                    ⁢          h1h2h3                =                  λ111          =                                    2              ⁢                              (                                                                            h1                      gx                                        ⁢                    cos                    ⁢                                          xe2x80x83                                        ⁢                                          α                      *                                                        +                                                            h2                      gy                                        ⁢                    cos                    ⁢                                          xe2x80x83                                        ⁢                                          β                      *                                                        +                                                            h3                      gz                                        ⁢                    cos                    ⁢                                          xe2x80x83                                        ⁢                                          γ                      *                                                                      )                                                                                      h1                  2                                                  gx                  2                                            +                                                h2                  2                                                  gy                  2                                            +                                                h3                  2                                                  gz                  2                                                                                        "LeftBracketingBar"        2        "RightBracketingBar"            
In the case of perpendicular incidence of light (xcex1xc2x0=xcex2xc2x0=90xc2x0, xcex=0xc2x0) a triple of chromatic diffraction orders results in the visible spectral region (380-780 nm)
xcex111 (longest wavelength) RED
xcex123 (average wavelength) GREEN
xcex122 (shorter wavelength) BLUE
The spectral transmission curves, which are centered relative to one of these xcexmax in each case, have a Gaussian shape and are determined at their half width by the number of the surface gratings in the z direction that are present in the 3D grating. In the event of incidence of white light, that is to say light of identical energy in all spectral components, and the grating inserted into the image plane of the imaging system, given the selection of xcex111=559 nm, the result is the trichromatism of the diffraction orders at
xcex111 RED=559 nm
xcex123 GREEN=537 nm
xcex122 BLUE=447 nm
There is thus a trichromatic tuning of the 3D grating, which is based on the resonant setting of the grating constants gx and gz to an integral xcex111, and in which a trichromatic equilibrium of the brightness values (Patterson amplitudes2 weights) is produced in the RGB diffraction orders.
In the case of adaptive chromatic retuning of the 3D grating to an illumination other than a white one, the relation of the RGB xcexmax (1 : 0.96 : 0.8 or 25 : 24 : 20) is always maintained. xcex111 is the resonant wavelength determining the triple shifts to shorter xcex111 wavelengths in the event of change to a blue illumination, and to longer xcex111 wavelengths in the event of change to a red illumination. The adaptive shift ends with the complete adaptation to the new illumination, that is to say with the resonant finding of a new RGB equilibrium, of the trichromatically additive white standard, which recenters the color space. The actual resonance factor is the phase velocity nvxcex=c (n=refractive index of the medium, v=frequency of the light, c=speed of the light).
The following new configuration of the sensor design forms the basis for the color constancy performance of the 3D grating optical sensor in the case of adaptation to variable illuminations.
A diffusion plate or disc or glass (hereinafter xe2x80x9cdiffusion glassxe2x80x9d) or one or more light diffusing gratings are incorporated into the pupil plane (aperture space) of the imaging optical system. Their function is to be seen in that they scatter diffusely as incoherent background into the image plane information likewise present at each location in the pupil, spatially the sum of the spectral intensities and local frequency values which are irradiated into the pupil by all objects in the object space and contribute to optical imaging. As a result, each local image location is supported by the global information on the entire field of view, against which each local pixel must stand out by being differentiated from it, specifically in brightness, hue, saturation etc. However, each item of local information thereby remains relativized in terms of the global background of the entire field of view.
All the lenses, diffusion glasses or gratings are designed such that they are transparent exclusively for electromagnetic radiation in the visible spectrum (380-780 nm) and therefore these delimit an octave of the wavelengths or frequencies with definite absorption edges. This boundary condition is important because thereby spectral brightness values which could come about through variation in the illumination are cut off at these absorption edges.
In the near field downstream of the diffractive 3D grating, the RGB interference maxima (3xc3x976 concentric maxima) assigned to a local pixel are interconnected via photoreceivers set in a constant fashion in terms of their spectral sensitivity to white sunlight (of identical energy in all spectral components) in such a way that a local RGB sum can be formed as a trichromatically additive brightness value by means of an appropriate evaluation. It is possible thereby to differentiate RGB equilibriums and disequilibriums. RGB equilibriums correspond in the object space to visible colorless surfaces or illuminations (black-gray-white objects). If illuminations are not visible, but can be inferred only via colorless objects or surfaces, they are represented in terms of their spectral characteristic by gray or white surfaces, what are termed mirrors of the illuminations. The image location with RGB equilibrium, which achieves the greatest aggregate brightness, supplies what is termed the white standard, and thereby defines the tip of the achromatic axis centering the color space. Alternatively, the image location whose RGB values most closely approximate an equilibrium takes over this provision of a white standard. This explains that the white standard can be displaced in the trichromatic space.
The design of a diffractive 3D grating optical sensor which provides trichromatic RGB values in three diffraction orders ensures color constancy when there is ensured together with the sudden or gradual change in the illumination in the object space a resonant mechanism, that is to say one that is adaptive to the spectral composition of the illuminations in the 3D grating, which corresponds to a chromatic tuning of the 3D grating. In the case of a white illumination, that is to say an equal-energy spectral composition, corresponding to average sunlight, of the visible light, three grating constants in the xyz-axial direction are tuned to the RED wavelength (559 nm), with standing wave formation in the x- and z-axial directions, that is to say resonance in the 3D grating. Identical values, that is to say RGB equilibriums, result thereby under the three Gaussian spectral photopic curves of the photoreceptors (cones in human vision). The white standard is determined via the RGB sum values of the three Gaussian curves, which are centered relative to the wavelengths 559 nm RED/537 nm GREEN/447 nm BLUE.
After a sudden or gradual change in the illumination, a chromatically triggered reconstruction of the grating constants takes place in the diffractive 3D grating. In the case of a displacement of illumination to the longer wavelength region of the spectrum, the white standard in the 3D grating, still tuned to 559 nm RED, suddenly breaks down. If the adaptive mechanism of the shifting of the white standard then acts in the direction of the changed illumination, the grating achieves a new RGB equilibrium in the case of a chromatic tuning to 728 nm RED, for example. The trichromatically additive color space is thereby centered again relative to an achromatic axis, and the colors are correct again, being experienced as correct.
If, by contrast, the illumination is displaced toward the shorter wavelength spectrum, the grating achieves a new RGB equilibrium, for example in the case of a chromatic tuning to 513 nm RED.
The adaptive process, which leads as a result to a trichromatic restandardization of the color space in a changed white standard, can be described by the already explained v. Laue equation of crystal optics. The actual resonance factor is the phase velocity in the medium. Spectral triggering of the grating constant dimensions corresponds to the coefficient of thermal expansion for the RED wavelength in the RGB triple. It is possible by means of dosed IR, that is to say thermal irradiation in the 3D grating, or by varying the internal pressure in the 3D grating to vary the grating constant dimensions correspondingly. The sensor according to the invention can thereby ensure the color constancy properties of the human vision system.
A color constancy sensor which is represented technically in the form of a 3D grating or preprocessing filter that can resonate with the centroid of the spectral component of a light source or illumination is very important for all applications in which the color of fabrics and materials must be detected, differentiated and classified by the color perception forming the basis of the laws of human vision. This also holds for the relevant judgment of properties of visible objects that are associated with the hue characteristics, whether this be in general image processing, whether with automatic viewers in robotics or autonomously driving vehicles, indeed even in the case of sensors for the blind. At the same time, such a sensor is able to render color perceptions under artificial light sources predictable and measurable. Since such a 3D grating transforms the physical parameters (intensity and wavelength) into the psychological triad of brightness, hue and saturation, it is also possible to use a sensor to calculate brightness and saturation values of object surfaces.